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ABSTRACT 


This paper investigates various approaches for automated inspection of 
gluing process using shape-based matching application. A new supervised 
defect detection approach to detect a class of defects in gluing application is 
proposed. Creating of region of interest in important region of object is 
discussed. Gaussian smoothing features is proposed in determining better 
image processing. Template matching in differentiates between reference and 
tested image are proposed. This scheme provides high computational savings 
and results in high defect detection recognition rate. The defects are broadly 
classified into three classes: 1) gap defect; 2) bumper defect; 3) bubble 
defect. This system does lessen execution time, yet additionally produce high 
precision in deformity location rate. It is discovered that the proposed 
framework can give precision at 95.77% recognition rate in recognizing 
imperfection for gluing application. 
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1. INTRODUCTION 

Vision-based inspection of industrial products offers low-cost, high-speed, and high-quality 
detection of defects. One of the most challenging industrial inspection problems is dealing with the textured 
of the gluing process. Defects often happen during gluing process. To ensure high quality units are produced, 
defects detection and recognition system commonly are installed for segregation of the defects unit. There are 
many techniques that provide a solution in recognizing image or object in image processing such as 
region [1], edge-based features [2], feature extraction [3], shape context [4], Gaussian Curve [5] and etc. The 
work of image recognition based on HALCON application for shape-based matching is done in [6-10]. 
The researches discussed about the process involved in basic shape based matching algorithm together with 
Extended Region of Interest (ROI) function available in HALCON that fulfils shape based matching to find 
object based on a single model image with sub pixel accuracy. The basic concept of defect matching using 
shape-based matching algorithm based on Extended ROI introduced by [11]. 

In this paper, an improve filter method that uses a computer to automatically recognize the gluing 
defects is utilized. In this approach, the digital image of the gluing defects is binarized first. Then ROI is 
further process before. Correlation algorithm refines the image. Once the importance feature of the binarized 
image has been extracted, the features are trained for defect classification. To better understand this method, 
in the next section, each kind of defects is elaborated further. ROI is applied to reduce processing time. 
Then depending on the features classification of the defect, feature extraction and Gaussian smoothing are 
used. Not only that, the position of the camera must be accurate in order to get better images [12]. 
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Minimal effort, rapid, and great discovery of deformities is the thing that vision-based assessment of 
modern items brings to the table. Issues managing the finished of the sticking procedure are the absolute most 
testing modern review. Deformities are regular happened in sticking procedure. Deformities identification 
and acknowledgment are most mainstream application used to affirm the uprightness of sticking procedure. 
Most scientists are for the most part unifying their reflection in welding line and furthermore in texture and 
many intriguing outcomes have been recovered [13-24]. For all intents and purposes, every one of these 
scientists can be delegated two sorts, one depends on radiographic investigation and the others dependent on 
man-made consciousness. The principal sort of strategies is to descry welding absconds physically, so the 
effectiveness is quite restricted [13-15]. The second sort of techniques are much outperforming in 
proficiency, anyway the unpredictability of execution is superfluous [16-24]. 

The other research is pinpointing on the welding imperfection [20, 24] that uses the component 
extraction strategy to rearrange picture to a straightforward calculation which depends on the perceptron 
model to descry and classify the lack as indicated by the information yielded from the extirpation technique. 
Picture of welding line is essential to the component destruction and deformity acknowledgment so as to 
perceive and characterize the welding deserts correctly. 


2. TYPES AND CHARACTERISTIC OF GLUING DEFECTS 

In this work, the study is performed to detect defects of gluing process at cardboard. Obviously, 
shape-based matching [25] requires reference image captured from the gluing process. The characteristic of 
each defect should be extract clearly as a model defect. The image obtained will undergo the pre-treatment of 
the images which include binarization processing, ROI, feature extraction, and Gaussian smoothing. Feature 
extraction processing is the important part that extracts the important features of each defect. This is because 
without proper feature extraction, the system cannot create high quality characteristic of the object. Without 
it, the recognition data will be affected. 

As all digital images of gluing defects are binary images for defects and background. The gray level 
of defects divided into 255, while the background area is 0. So when the gray level of all pixels of an image 
is 0, then there are no defects in the gluing process based on the image. There are three kinds of defects 
altogether in gluing process which are gap defect, bumper defect and bubble defect. To begin with, the 
characteristic of these defects will be analyzed which will benefit for feature extraction as shown in Table 1. 


Table 1. The Characteristic of Each Defect 


No. Defect Description 
1 Gap Defect Gluing line surface that irregular, parallel empty space with gluing line, glue that 
rugged until exposing space that not encrusted with glue. 
2 Bumper Defect Gluing line surface that irregular, swollen that similar to hump until deform 
gluing line surface due to imbalance in object surface. 
3 Bubble Defect Gluing line surface that regular but is trapped air bubble in gluing line which 


caused object surface not glue perfectly. 


3. OVERALL SYSTEM 

One of the applications of this research is to provide improved gluing car windscreen in automation 
industry. The system is developed to cope with the environment such as lighting condition, scaling and 
rotating of the object. Pyramid object developed from plain cardboard is created based on the shape object 
that has a contour that involves height, length and width. Plain cardboard is design exactly like pyramid to 
get the data from the Defect Shape Matching (DSM) whether the system able to recognize the object. 
Two cameras are used to locate x, y and z location for each corner detected through the system. Equation of 
straight line will be used in determining additional point between two corners. Data gathered must be 
converted to robot coordinate for easier process by Katana and KUKA arm robot. This information then used 
to instruct robot for gluing purposes. Three types of defect are trained through Defect Inspection Monitoring 
(DIM) known as gap defect, bumper defect and bubble defect. Result obtained from DIM will decide whether 
the object is successful gluing or needs to be corrected through Correction of Defect (CoD). Figure 1 show 
the overall system of this research. 
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Figure 1. Proposed vision system 


4. SHAPE-BASED MATCHING 

Shape-based matching application divides into two phases namely as training phase and recognition 
phase [10] as shown in Figure 2. Training phase is a crucial part in identifying accurate defectiveness. 
Optimized training system will results in less accurate matching performance. Therefore, accurate matching 
results are depending to high quality reference image. The recognition phase is performed to find the 
similarity between reference images and tested images, where the similarity index in known as recognition 
rate. High recognition rate reflecting that, the proposed system is effective and optimized. Shape-based 
matching system is always favorable since the user can simply choose the desired region, fast processing 
time and provide higher recognition rate. 

The proposed method consists of 4 main steps that valid in succession of classification glue defects: 
1) Feature Extraction; 2) Gaussian smoothing; 3) Template creation; 4) Template matching. Introduction to 
this system further increase recognition rate and further accelerate inspection process. Furthermore, 
this proposed method develops with simplest features that only focused on images as an input and output. 
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5. FEATURE EXTRACTION 
To classify and recognize the defects mentioned Section 2, three features have been adopted. 
They are gray level, region of interest and converting skeleton to XLD contours. 


5.1. Gray Level 
To classify and recognize the defects mentioned above, three features has been adopted. There are 
gray levels, ROI and skeleton to XLD contours conversion. The grey level can be expressed as the following. 


gray = 0.299a + 0.587ß + 0.144y (1) 
Where a, P and y are red, green and blue respectively. 


This feature affords in separating colors that may influence defect results. Gray level application 
selection helps in image of interest isolation with object background. The results facilitate system in 
developing defect model template. 


5.2. Region of Interest 

Region of interest (ROI) can be defined as a selected subset of samples within the dataset identified 
for a particular purpose known as boundary of an object. ROI provides the object to be match and its 
background as an input. This feature helps to minimize noise, improves matching speed and reduces 
matching parameter of the system since only the data within the selected ROI is used for the next process. 


5.3. Converting Skeleton to XLD Contours 

Changes over the info skeleton (e.g., edges), which is accepted that contain generally one-pixel wide 
areas, into XLD shapes. The district is first changed to contain just line fragments in 8-neighborhood. In the 
process 12 exceptional designs are considered: Focuses for which there is an intersection of at least three 
lines in 8-neighborhood is saved (in every one of the four turns):: 


00 1\ /0 1 0\ /0 1 0 
110] {0 1 1] {111 (2) 
0 1 0 \o 1 0/ \o 1 0 


In a subsequent advance, all intersection focuses are labeled, considering six distinctive trademark 
setups of each of the four potential pivots: 
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Where, 0 = background, 1 = foreground and 2 = junction point. 

Consequences of the two stages are for the most part forms having in any event length focuses are 
returned. The shapes created consistently end in intersection or end focuses. For shut shapes the main point 
lies in the 8-neighborhood of the last purpose of the form. In this manner, so as to decide the contiguousness 
of forms it is adequate to simply consider the end focuses. 

The importance of this step provides better shape of the defects model. Not only that, 
time consuming in classified the defect may reduce. 
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6. GAUSSIAN SMOOTHING 

The smoothing of gray images are using recursive and non-recursive Gaussian filter. There is 
available in four types of filter: 1) derichel; 2) deriche2; 3) shen; 4) gauss. The channel width (1.e., the scope 
of the channel and in this manner consequence of the channel) of the picture can be of any size. For the 
situation that he Deriche or Shen is picked it diminishes by expanding the channel parameter alpha (a) and 
increments on account of the Gauss channel (and a compares to the standard deviation of the Gaussian 
capacity). An estimate of the proper size of the channel width is performed. 

Non-recursive channel like the Gaussian channel are regularly executed utilizing channel veils. 
For this situation, the runtime of the administrator increments with expanding size of the channel cover. 
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The runtime of the recursive channels stays steady: with the exception of the outskirt treatment turns into 
somewhat additional tedious. A practically identical aftereffect of the smoothing is accomplished by picking 
the accompanying qualities for the parameter: 


a(deriche1) 


a(deriche2) = > (4) 
_ a(deriche1) 

a(shen) = — ~~ (5) 

a(gauss) = 2 (6) 


a(deriche1) 


7. TEMPLATE CREATION 

After image extracted using feature extraction and Gaussian smoothing, template creation is 
performed. Template creation is important in determining the originality of each defect characteristic to be 
submitted into the system for matching purposes. During the template creation step, the size of the required 
features must be considerably large to prevent the created outline does not overlap with the object and 
background can cause inaccuracy. 

In order to achieve good template matching results, the pixel value difference between the object 
and the background must be sufficient for the system to work effectively. During the training phase, 
the resultants template must have an obvious shape. The training templates for this experiment are shown as 
example in Figure 3. 


< D D 





Template Gap Defect Template Bumper Defect Template Bubble Defect 


Figure 3. Template image for each defects 


8. TEMPLATE MATCHING 

Template Matching is a procedure in computerized picture handling for discovering little pieces of a 
picture which match a template picture. From the boundary pattern available, a direct template matching 
algorithm can be applied. However, such method requires huge amount of computation. In order to minimize 
it, the conversion of 2D image object location into single dimension must be performed. Therefore, centroid 
features are proposed. 


8.1. Centroid Profiles 

Centroid is the center of gravity of the object’s shape. From the geometric coordinate values (x,y) of 
the boundary pattern, the centroid can be calculated using average values of these coordinate values. 
The centroid algorithm is shown in (7). The centroid for x-plane and y-plane is calculated separately and the 
resulting value will be center coordinate. 


_ x1+x2+--+xk 
k 


X (7) 

The centroid of a finite set of points x1, x2, ..., xk, x is where k is the number of pixels. In template 
matching, model template played an important role in classifying each defect. Without proper characteristics 
of defects, the matching process ends with no results. Therefore, the planning of the system should integrate 
with both features combining in providing better recognition rate. 
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9. DISCUSSION OF RESULTS 

The points of this work are to show an adaptable visual framework for shape based matching. 
Test results are utilized to check the proposed methodology. In this test, three imperfections models and their 
relating tests are utilized to look at this methodology. Table 1 demonstrates the comparing results as 
indicated by 10 tried pictures utilized. Each tried picture has its own deformity to be perceived by this 
framework. Every one of the information results are recorded into Table 1. 

Figure 4 shown to express the step by step results in this system. Figure 4(a) to Figure 4(c) shows 
the result of each defect model having through the 4 steps of training phase. Each defect model has its own 
characteristic in classifying defects. Figure 4(d) to Figure 4(f) shows the result by matching model defects on 
the tested images. By using template matching, each pixel of the model defects is compared with tested 
images in finding the exact similarity with it. Figure 4(g) to Figure 4(1) shows the gray histogram of each 
model defect. 


a 





(c) 
Figure 4. Result of each defect from model template, template matching and gray histogram 
Table 2 shows the result of defect matching by using 10 tested images. According to the results, 
the recognition rate of the experiment is about 95.77% based on 3 model defects created through the system. 


This result is compared with [11] where it considered the problem in detecting welding defects in welding 
lines where the past researchers put an effort on more complex algorithm or limited in efficiency. 
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Table 2. Matching Results from 10 Tested Images 


Image Total Defect Defect Detected Defect Detected (%) 
l 8 8 100 
2 7 7 100 
3 8 7 87.5 
4 7 6 85.71 
5 8 8 100 
6 6 6 100 
7 7 7 100 
8 8 8 100 
9 5 5 100 
10 7 6 85.71 

Total 71 68 95.77 


10. CONCLUSION 

As a conclusion, auto recognition of sticking deformities is a helpful research with solid application 
foundation. In this paper, a technique that is utilized to concentrate highlights of imperfections and auto 
perceives deformities are advanced. Trial results demonstrate that the acknowledgment rate accomplished 
about 95.77% that greatly improved for gaining deformity discovery in sticking lines. For the most part, 
the technique for acknowledgment and order of imperfections in sticking lines are compelling. Furthermore, 
it would much be able to decrease the working exertion of person and increment the imperfection 
acknowledgment proficiency. 
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